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(57) Abstract 

A method and apparatus for dy- 
namically allocating the available band- 
width of a common transmission channel 
of a multiplexed system among multiple 
encoders in such a manner as to maxi- 
mize and equalize the quality of the en- 
coded data output by all of the encoders, 
while also preventing underflow or over- 
flow of encoder or decoder buffers at 
each end of the common transmission 
channel, and moreover, while also ensur- 
ing compliance with (i.e., without violat- 
ing) the data encoding and transmission 
protocol utilized by the system. Further, 
the bandwidth of the common transmis- 
sion channel is preferably allocated us- 
ing an algorithm that docs not impose 
any constraints on the size of the en- 
coder or decoder buffers, other than any 
constraints specified by the data trans- 
mission protocol employed in transmit- ^22 
ting the encoded data over the common 

fe^i^ed^^r^b^^^ Shich^lS^sfgnals from multiple sources are encoded, both the output channel rale (■'bit rate") of each of the encoders 
and the target number of bits for each picture that is encoded by each encoder are controlled by a controller m accordance with a control 
alEorithm which ensures that the quality of the encoded pictures output by all of the encoders is equalized and maximized, that no underflow 
or overflow of the encoder or decoder buffers occurs, and that the data encoding and transmission protocol utilized by the system is not 
violated. Further, the control algorithm employed by the controUer preferably does not impose any constraints on the sizes or relative sizes 
of the encoder or decoder buffers. Moreover, the control algorithm employed by the controller preferably maintains all of the encoder 
buffers as empty as possible in order to provide increased flexibility to the rale allocation procedure. 
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1 

Method and apparatus for dynamically controlling encoding parameters of multiplexed 
encoders in a multiplexed system 

BACKGROUND OF THE INVENTION 

The present invention relates generally to data encoding and transmission 
systems in which data from multiple channels are multiplexed into a single data stream, and, 
more particularly, to a system of this type in which encoding parameters of the encoders for 
the respective multiple channels are dynamically controlled by a controller. A generally 
related invention by the present inventor is disclosed in co-pending U.S. Patent Application 
Serial Number 08/214.910, filed March 17, 1994, and entitled "AN ENCODER BUFFER 
HAVING AN EFFECTIVE SIZE WHICH VARIES AUTOMATICALLY WITH BIT 
RATE". The disclosure of this application, which is also assigned to the assignee of the 
present invention, is herein incorporated by reference. 

In multiplexed data transmission systems, data from a plurality of 
different data sources, or data channels, are combined or multiplexed, at a head end station, 
into a single data stream for transmission over a transmission link, such as a wire, fiber optic 
or radio link, to a back end station, where the channels of data from the multiplexed data 
stream are then separated, or demultiplexed, and supplied to intended recipients. 

In an exemplary system of particular interest herein, a plurality of video 
signals from different video sources, such as different television network feeds, different 
television stations, or other video sources, are transmitted over a satellite link for direct 
broadcast to respective television receivers ("set-top boxes") located in the homes of 
individual consumers. An exemplary satellite link includes a digital transmission path 
capable of transmitting 24 megabits per second (Mbps). In order to maximize the data 
transmission efficiency and utilization of such a link (i.e., in order to maximize utilization of 
the available bandwidth), it is necessary for several video signals (e.g., six video signal 
channels) to share the link (i.e., to be transmitted as a single, multiplexed data/bit stream). 

In a typical multiplexed video signal transmission system, the video 
signals from the different sources are encoded by respective parallel variable bit rate (VBR) 
video encoders as successive fields or frames ("pictures"), and the encoded video signals 
output by the video encoders are combined into a single digital video bit stream for 
transmission over a common channel (e.g., the direct broadcast satellite (DBS) link) by a 
multiplexer. In general, the number of bits required to encode a picture of a given quality 
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increases in accordance with the spatial and temporal complexity of the picture. Thus, 
pictures corresponding to spatially complex scenes and/or scenes with motion require more 
bits to encode them than do pictures corresponding to spatially simple scenes with little 
motion. In this regard, the quality of pictures of different complexity can be equalized by 
5 coding the pictures using a number of bits which is functionally related to the complexity of 
the picture. 

It is generally assumed that, statistically, the average combined bit rate 
from all of the parallel video encoders will be less than the maximum bit rate of the common 
transmission channel allowed by the transmission protocol, even though at any given time a 
10 single video encoder (for a single channel of video signals) may provide a large number of 
bits in a burst for transmission. For this reason, such apparatus is commonly referred to as a 

"statistical multiplexer". 

However, there is a finite probability that the instantaneous combined bit 
rate from all of the channels (i.e., the combined bit rate of the encoded video signals output 

15 by the parallel video encoders) will exceed the maximum allowable bit rate of the common 
transmission channel, thereby resulting in a loss of data (picture information) at the back end 
station (e.g. , consumer television set). In order to minimize such occurences, encoder and 
decoder buffers are utilized. In this regard, most video coding standards (e.g., MPEG) 
specify the decoder buffer size. However, there still exists a finite probability that the 

20 encoder and/or decoder buffers will overflow (or underflow), again resulting in loss of data 

at the back end station. 

In general, a significant goal of system designers is to reduce the required 

size of the decoder buffers (without unduly compromising picture quality), since decoder 

buffers are an important component of the cost of the consumer set-top boxes. On the other 
25 hand, the size of the encoder buffers is generally not a major constraint in the design of the 

system, since the cost of the encoder buffers is a relatively insignificant component of the 

overall cost of the head end station. 

Various systems have been proposed in order to maximize the utilization 

of the bandwidth of the common transmission channel while maintaining acceptable picture 
30 quality and ensuring compliance with a digital video data encoding, transmission, and 

decoding protocol (e.g.. an MPEG protocol). Several of these proposed systems are 

discussed below by way of background. 



European Patent Application No. 92121030.8, published on December 10, 
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1992, in the name of Paik et al., discloses a statistical multiplexer which uses a common 
global control parameter that determines the quality of the encoded pictures output by each of 
the video encoders. However, Paik et al. do not provide a solution for allocating bit rate 
among the video encoders. Therefore, since the complexity of the pictures from each video 
5 source are not taken into consideration, the quality of the encoded pictures output by the 
different video encoders can vary widely, e.g., from poor quality to excellent quality, 
depending upon the complexity of the picture. 

U.S. Patent Number 5,115,309. issued on May 1992 to Hang, discloses a 
method and apparatus for dynamic channel bandwidth allocation among multiple parallel 

10 video coders for coding a single picture. However, the method and apparatus disclosed by 
Hang is not appplicable to parallel coding multiple pictures, 

An article entitled ''Multichannel Joint Rate Control of VBR MPEG 
Encoded Video for DBS Applications", IEEE Transactions on Consumer Electronics, August 
1994, by Guha and Reininger, discloses a multiplexed system in which large video encoder 

15 buffers are employed. The encoded picture bitstream produced by each video encoder is 
stored in its associated large encoder buffer, and the required average channel rate for 
transmitting the same is calculated. If the required average channel rate exceeds a prescribed 
maximum channel rate, the video data is re-coded in such a manner as to reduce the average 
channel rate by an appropriate amount, 

20 An article entitled "Statistical Multiplexing of Multiple MPEG-2 Video 

Programs in a Single Channel", SMPTE Journal, September 1995, by Perkins and Arnsiein, 
discloses a multiplexed system that dynamically allocates bit rates to the different channels 
(i.e., different parallel video encoders) based on the complexity of the pictures currently 
being transmitted for all of the channels. However, the disclosed system requires that each 

25 video encoder buffer be smaller than its corresponding video decoder buffer by a factor that 
is equal to the ratio of the maximum and minimum chaimel rates of the encoder. This 
presents an undesirable trade-off between the required decoder buffer size and the quality of 
the encoded pictures. More particularly, the quality of the picture coded by an encoder 
buffer is related to the size of the buffer. Since the encoder buffer must be smaller than its 

30 corresponding decoder buffer in accordance with the above-stated constraint, then reducing 
the required size of the decoder buffer results also results in a reduction in the size of the 
encoder buffer, thereby resulting in degraded picture quality. 

In this connection, as previously mentioned, most video coding standards 
(e.g., MPEG) specify the minimum decoder buffer size. For an MPEG-2 MP@ML system. 
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the video decoder buffer size is specified to be 1,835,008 bits. If the encoder operates at bit 
rates ranging from 1.5 Mbps- 15Mbps, the size of the encoder buffer according the above- 
stated buffer size constraint must be 183.500 bits. If the range of bit rates is 1.5 Mbps-6 
Mbps, the encoder buffer size must be more than 458,752 bits. In either case, the video 
5 quality is signficantly compromised. In the second case, the maximum operational rate is 
also reduced. 

PCT Application Number WO 95/29559, published on November 2, 
1995. in the name of Ozkan et al., also discloses a multiplexed system that dynamically 
allocates bit rates to the different channels (i.e., different parallel video encoders) based on 

10 the complexity of the pictures currently being transmitted for all of the channels. The 

complexity of the pictures currently being transmitted for all of the channels are evaluated, 
and a proportion of the total bit rate of the transmission link is allocated to each channel 
corresponding in some manner to the relationship of the complexity of the current picture of 
that channel to the overall complexity of the pictures of all of the channels, such that, during 

15 any time period, the quality of the coded pictures of all of the channels is approximately the 
same. Thus, a channel transmitting a high complexity picture for a period of time will be 
dynamically allocated a higher bit rate during that period of time, but when the complexity of 
the picture becomes lower, the bit rate allocated to that channel will be reduced, and then re- 
allocated to other channels which are transmitting higher complexity images. 

20 The Ozkan et al. multiplexer system suffers from the same drawbacks 

discussed above in connection with the Perkins and Arnstein system regarding the required 
trade-off between picture quality and decoder buffer size. In this regard, it is believed that 
the required size of the decoder buffers in the Ozkan et al. multiplexer system for pictures of 
acceptable quality is so large as to render the decoder buffers cost-prohibitive for consumer 

25 and other applications, and, in fact, so large as to possibly not comply with MPEG decoder 

buffer size requirements. 

Moreover, the Ozkan et al. multiplexer system unnecessarily limits the 
maximum bit rate change which can be made between successive coding periods. This 
constraint results in an inability of the system to dramatically change the allocated bit rate in 
30 response to dramatic scene changes, which in turn results in poor picture quality for pictures 
corresponding to dramatic scene changes. 



Based on the above and foregoing, it can be appreciated that there 
presently exists a need in the art for a method and apparatus tor dynamically controlling 
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encoding parameters of multiple encoders in a multiplexed system which overcomes the 
above-discussed drawbacks and shortcomings of the presently available technology. The 
present invention fulfills this need in the art. 

5 SUMMARY OF THE INVENTION 

The present invention encompasses a method and apparatus for 
dynamically allocating the available bandwidth of a common transmission channel of a 
multiplexed system among multiple encoders in such a manner as to maximize and equalize 

10 the quality of the encoded data output by all of the encoders, while also preventing underflow 
or overflow of encoder or decoder buffers at each end of the common transmission channel, 
and moreover, while also ensuring compliance with (i.e., without violating) the data 
encoding and transmission protocol utilized by the system. Further, the bandwidth of the 
common transmission channel is preferably allocated using an algorithm that does not impose 

15 any constraints on the size of the encoder or decoder buffers, even if the size of the decoder 
buffer is specified by the data transmission protocol employed in transmitting the encoded 
data over the common transmission channel. 

In a presently preferred embodiment of the present invention, in which 
video signals from multiple sources are encoded, both the output channel rate ("bit rate") of 

20 each of the encoders and the target number of bits for each picture that is encoded by each 
encoder are controlled by a controller in accordance with an algorithm which ensures that the 
quality of the encoded pictures output by all of the encoders is equalized and maximized, thai 
no underflow or overflow of encoder or decoder buffers occurs, and that the data encoding 
and transmission protocol utilized by the system is not violated. Further, the algorithm 

25 employed by the controller preferably does not impose any constraints on the sizes or relative 
sizes of the encoder or decoder buffers. Moreover, the algorithm employed by the controller 
preferably maintains all of the encoder buffers as empty as possible in order to provide 
increased flexibility to the rate allocation procedure. 

In one of its aspects, the present invention embraces a multiplexed data 

30 transmission system which includes a plurality of encoders each having a data input coupled 
to a respective one of a plurality of data signals, a control input, and an encoder output, a 
multiplexer having a plurality of multiplexer inputs coupled to respective ones of the encoder 
outputs, and a multiplexer output coupled to a transmitting end of a data transmission 
channel, and, a controller for producing control signals in accordance with a control 
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algorithm, the control signals being coupled to the control input of respective ones of the 
encoders for dynamically controlling at least two encoding parameters of the encoders, a first 
one of the encoding parameters being an output bit rate of each of the encoders. The 
multiplexer operates to multiplex the encoder outputs and to output a single, multiplexed 
5 encoded data stream over the data transmission channel at a transmission channel bit rate 
which is less than or equal to a prescribed maximum transmission channel bit rate. The 
transmission channel bit rate is equal to the sum of the output bit rates of all of the encoders. 

The second one of the encoding parameters is preferably a target number 
of bits used by each of the encoders to encode each of a plurality of prescribed blocks of 
10 data contained in respective ones of the data signals. 

The control algorithm preferably ensures that the quality of encoded data 
represented by all of said encoder outputs is substantially equalized without violating a data 
encoding and transmission protocol utilized by the system, e.g.. an MPEG protocol. The 
control algorithm preferably imposes no constraints on the allowable sizes or relative sizes of 
15 the encoder or decoder buffers located on either side of the data transmission channel. The 
control algorithm also preferably minimizes the number of bits contained by each of the 
encoder buffers. The control algorithm also preferably ensures that no underflow or 
overflow of either the encoder buffers or the decoder buffers occurs. 

In a presently preferred embodiment of the present invention, the data 
20 signals are video signals from a plurality of different video sources, whereby the prescribed 
blocks of data comprise successive picmres contained in respective ones of the video signals. 
The target number of bits are dynamically allocated by the controller in accordance with the 
control algorithm in such a manner that all of the encoders are allocated a target number of 
bits proportional to a measure of complexity of a picture currently being encoded thereby, 
25 The control algorithm preferably imposes an additional constraint that an actual number of 
bits used by each of the encoders in encoding each of the pictures not be greater than a 
dynamically allocated maximum number of bits and not be less than a dynamically allocated 
minimum number of bits. 

30 The controller preferably dynamically varies the output bit rate of each of 

the encoders simultaneously at prescribed bit rate change intervals, in accordance with the 
control algorithm, and dynamically varies the target number of bits used by each of the 
encoders on a piciure-by-piciure basis, in accordance with the control algorithm. 

Each of the encoders preferably communicates to the controller the actual 
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number of bits used thereby in encoding each picture encoded thereby, as well as a measure 
of complexity of each picture encoded thereby. In this regard, the controller most preferably 
dynamically varies the output bit rate of each of the encoders simultaneously every 1/F 
seconds, in accordance with the control algorithm, where F is a maximum frame rate of any 
of the video signals. 

The present invention also embraces a method generally corresponding to 
the above-described apparatus of the present invention. 

BRIEF DESCRIPTION OF THE DRAWINGS 



These and other objects, features, and advantages of the present invention 
will become apparent from the following detailed description taken in conjunction with the 
accompanying drawings, in which: 

FIG. 1 is a block diagram of a multiplexed system constructed in 
15 accordance with a presently preferred embodiment of the present invention; 

FIG. 2 is a block diagram of a simple model of an end-to-end system 
configuration that is used in the derivation of the control signals CSi that are generated by 
the controller of the multiplexer system of the presently preferred embodiment of the present 
invention; 

20 FIG. 3 is a diagram which graphically depicts the timing of rate events 

and coding events for different encoding channels of the multiplexer system of the presently 
preferred embodiment of the present invention; and, 

FIG. 4 is a diagram which graphically depicts the changing of the bit rate 
of an encoder i of the multiplexer system of the presently preferred embodiment of the 

25 present invention, 

DETAILED DESCRIPTION OF THE INVENTION 

Although the present invention will be described below in terms of a 
30 presently preferred embodiment thereof, it should be clearly understood that the present 
invention is not limited, in its broadest sense, to this particular embodiment or 
implementation of the basic inventive concepts herein taught. For example, although the 
present invention is described herein in terms of a multiplexed system which processes video 
signals, it should be clearly understood that the basic inventive concepts disclosed herein are 
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not limited to the processing of video signals, but rather, are equally applicable to the 
processing of any other type of data to be encoded and transmitted, and subsequently 
decoded, e.g., audio, text, voice, fax, and other forms and types of data. 

With reference now to FIG. 1, there can be seen a block diagram of a 
multiplexed system 20 constructed in accordance with a presently preferred embodiment of 
the present invention. As can be seen, video signals Vi from a plurality of different sources 
(e.g., different television stations and/or video feeds) are applied to the video input terminal 
V of respective, parallel video encoders 22, the respective channel outputs of which are 
combined by a multiplexer 24 into a single coded bit stream for transmission over a single 
transmission channel 26 (e.g.. a DBS downlink) to a receiving station 28 (e.g.. a set-top box 
in a home of an individual consumer). The receiving station 28 typically includes a 
demultiplexer for demultiplexing the bit stream in order to separate the bit stream into its 
constituent channels of encoded video data, which are then supplied to respective video 
decoders. As will be described more fully hereinafter, each of the video encoders and video 
decoders includes an associated buffer (see FIG. 2) for buffering the encoded and decoded 
video data, respectively. It is presently contemplated that the present invention will find 
particular utility in a DBS system in which the video signals from the different sources are 
encoded, transmitted, and decoded in accordance with the MPEG-2 T-STD and/or VBV 
models. Of course, this particular application is not limiting to the present invention. 

In accordance with the present invention, the multiplexed system 20 
includes a controller 30 (e.g., a microprocessor, microcontroller, or other suitable digital 
signal processing device) which is programmed to allocate the available bandwidth of the 
transmission channel 26 to the multiple video encoders 22 in accordance with a control 
algorithm which ensures that the quality of the encoded pictures output by all of the video 
encoders 22 is equalized and maximized, that no underflow or overflow of encoder or 
decoder buffers occurs, and that the data encoding and transmission protocol utilized by the 
system is not violated. In this regard, the channel output rates (i.e., bit rates) for each 
encoder 22 are chosen by the controller, in accordance with the control algorithm, in such a 
way as to ensure that each encoder 22 will be able to continue to operate without causing 
decoder or encoder buffer overflow, even if a failure of the comroller 30 (or other system 
"failure mode") prevents the bit rates of the encoders 22 from being dynamically controlled. 
It will be appreciaed by those skilled in the pertinent art that the bit rates of the encoders 22 
can be effectively varied by either controlling the rate at which the encoders 22 "push" the 
encoded data to the multiplexer 24. or. by controlling the rate at which the multiplexer 24 
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"pulls" the encoded data from the encoders 22, 

Further, the control algorithm employed by the controller 30 preferably 
does not impose any constraints on the sizes or relative sizes of the encoder or decoder 
buffers. Moreover, the control algorithm employed by the controller 30 preferably maintains 
5 all of the encoder buffers as empty as possible in order to provide increased flexibility to the 
rate allocation procedure, all as will become more fully apparent hereinafter. 

More particularly, the controller 30 generates control signals CSi which 
are applied to the control signal input terminal C of respective ones of the parallel video 
encoders 22 for controlling both the channel output rate {i,e., "bit rate") of each respective 

10 encoder 22 and the target number of bits to be used by each respective encoder 22 in 

encoding each picture (e.g., field or frame of the input video signal Vi). It will be readily 
appreciated by those skilled in the pertinent art that the present invention separately controls 
two different encoding parameters of each of the video encoders (i.e., bit rate and target 
number of bits to be used in encoding each picture), rather than simply controlling just a 

15 single parameter (i.e., bit rate), as has been done heretofore. As will become more fully 
apparent hereinafter, this separate control of two different encoding parameters enables the 
above-stated goals of the present invention to be realized. 

With reference now to FIG. 2, there can be seen a block diagram of a 
simple model of an end-to-end system configuration that is used in the derivation of the 

20 control signals CSi that are issued by the controller 30 to an exemplary video encoder 22i. 
This model is used in order to enable implementation-specific details of the video encoder 
22i and the controller 30 to be handled independently. As can be seen, the video encoder 
22i on the transmitting end includes a preprocessing unit 23, a compression engine 25. and a 
logical buffer 27, and the video decoder 33i on the receiving end includes a buffer 37, a 

25 decompression engine 39, and a postprocessing unit 41, Further, the coded output of the 
video encoder 22i is applied to an input of a multiplexer 24', and the multiplexed coded 
bitstream which is transmitted over the transmission channel 26 is applied to an input of a 
demultiplexer 43 whose output is coupled to the input of the video decoder 33i, 

The input video signal Vi applied to the preprocessing unit 23 and the 

30 output video signal Vout' from the postprocessing unit 41 are either interlaced or 
progressive, and are both produced at a suitable frame rate, e.g., 25, 29.97, or 30 
frames/ second. The successive pictures (either fields or frames), indexed by j, of the input - 
video signal Vi, enter the compression engine 25 of the video encoder 22i at respective 
encode times tc(i,j). Each picture j of the input video signal Vi is instantaneously 
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compressed and all of the compressed data which constitutes that picture j is input to the 
logical buffer 27 of the encoder 22i instantaneously, at time t,(i.j). As will be appreciated by 
those skilled in the pertinent art, the logical buffer 27 shown in FIG. 2 is not the physical 
buffer of the encoder 22i, which, can be much larger. 

5 The rate at which the encoded video data (i.e., compressed/coded picture 

data) is ouipui from the video encoder 22i at a time l is specified as R,(t). Further, assuming 
that there is zero delay in the network/transmission channel 26", at time t, the demultiplexed, 
encoded video data enters the buffer 37 of the video decoder 33i at the same rate R,(t) that 
the data is output from the logical buffer 27 of the video encoder 22i. All of the compressed 

10 data which constitutes the picture j is then output from the buffer 37 of the video decoder 33i 
at a time tj(i.j) specified by the decode time stamp (DTS) of the picture, for decompression 
by the decompression engine 39. 

For purposes of simplifying the model depicted in FIG. 2 for illustrating 
the derivation of the control signals CSi that are issued by the controller 30. the following 

15 assumptions are made: 

(a) in the case of an MPEG-2 system, only elementary stream data is present in the logical 
buffer 27 of the video encoder 22i, i.e.. there is no PES Header data in this buffer; 

(b) t,(i.j) = A,, where A, is the end-to-end buffer delay and is assumed to be a constant for 
each encoder 22i. it being understood that the actual buffer delays are usually different for 

20 the different encoders 22 i; 

<c) data transfer between the logical buffer 27 of the video encoder 22i and the multiplexer 
24- at time i is based on the buffer transfer rate R,(t), which is assumed to be a piecewise 
constant function; and, 

(d) deviations from the illustrative model depicted in FIG. 2 are handled internally of either 

25 the encoder 22i and/or the multiplexer 24'. 

Some deviations from the illustrative model depicted in FIG. 2 are that 
are present in existing encoder systems include the following: 

(a) the encoder 22i does not instantaneously insert all of the data that comprises a picture j 

into its logical buffer 27; 
30 (b) the data that is transferred from the logical buffer 27 of the encoder 22i to the 

multiplexer 24' may include packetization overhead (i.e., overhead data required by the 
protocol, as opposed to strictly pay load data); and, 

(c) depending on how the encoder 22i handles the use of the repeat_first_rield feature (for 



wo 98/11687 



PCTAIB97^01085 



performing a ihree-iwo pulldown procedure on the incoming video data Vi), the end-to-end 
buffer delay may not be a constant. As will be readily appreciated by those skilled in the 
pertinent art, these and other possible deviations from the illustrative model depicted in FIG. 
2 can be handled internally of the encoder 22i and/or the multiplexer 24*. 
5 In the presently preferred embodiment of the present invention lo be 

described in greater detail hereinbelow, the control algorithm implemented in software 
residing in the controller 30 is designed to ensure that: 

(i) the coded bitstream produced by each video encoder 22 will be compliant with the 
MPEG-2 T-STD and VBV models; 
10 (ii) the buffer 27 for each video encoder 22 will not underflow or overflow; 

(iii) the buffer 37 for each video decoder 33 will not underflow or overflow; 

(iv) the sum of the output rates (bit rates) of all of the encoders 22 will not be greater than 
the maximum allowable channel rate (i.e, the maximum bit rate of the transmission channel 
26); and, 

15 (v) the perceived quality of the pictures across all channels will be equal (and optimized). 

For purposes of the ensuing further detailed description of the presenly 
preferred embodiment of the present invention (and for the appended claims), the following 
notations will be utilized: 

(i) i = 1 through L, i.e., a total of L video encoders 22 are included in the multiplexed 
20 system 20; 

(ii) the size of the buffer 37 of a decoder 33i that is receiving the data generated by the 
corresponding encoder 22i is B^ ,; 

(iii) the size of the buffer 27 of an encoder 22i is B^,,; 

(iv) the number of bits contained in the buffer 27 of an encoder 22i at lime t is Ei(t); 

25 (V) the number of bits contained in the buffer 37 of the decoder 33i receiving data produced 
by the corresponding encoder 22i is D,(t); 

(vi) the number of bits used in encoding the jth picture that is encoded by an encoder 22i is 
b(ij); 

(vii) the output rate (bit rate) of an encoder 22i is Ri(t). This rate varies with time, but is 
30 piecewise constant; 

(viii) the end-to-end buffer delay of an encoder 22i and its corresponding decoder 33i is A.; 

(ix) the target number of bits allocated for encoding the next I picture to be encoded by an 
encoder 22i is T,i(t). The target number of bits T,i(t) also varies with time. Thus, if the next 
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picture I is encoded at lime t . the target number of bits allocated for encoding the next I (I- 
type) picture will be T,i(T); 

(X) the target number of bits allocated for encoding the next P (P-type) picture to be encoded 
by an encoder 22i is Tpi(t); 
5 (xi) the target number of bits allocated for encoding the next B (B-type) picture to be 
encoded by an encoder 22i is TeiCt); 

(xii) the estimate at time t of the complexity of the next I (I-iype) picture of the video 
sequence being encoded by an encoder 221 is Xn(t). The complexity of the video sequence 
also varies with lime. The actual measure of complexity utilized in the practice of the 

10 present invention is not critical or limiting thereto. The only requirement is that the measure 
of complexity utilized be of a nature that it is meaningful to compare these measures across 
video sequences, in order to provide a meaningful indication of the relatively complexity of 
the pictures being encoded by the different encoders 22; 

(xiii) the estimate at time t of the complexity of the next P picture of the video sequence 
15 being encoded by an encoder 22i is Xpi(t); 

(xiv) the estimate at time t of the complexity of the next B picture of the video sequence 
being encoded by an encoder 22i is X^M- Although the values of the three parameters 
Xiid), Xpi(i), and XBi(t) can vary with time, they will typically change substantially only at 
scene changes in the incoming video signal; 

20 (XV) the GOP ("group of pictures") size for each encoder 22i is N^t). The value of this 
parameter can vary with time; 

(xvi) the M parameter for an encoder 22i is M,{i). The value of this parameter can also vary 
with time; 

(xvii) the picture rate at time t for an encoder 22i is Hi). The picture rate of an encoder can 
25 vary with time as it transitions in and out of film-mode; and, 

(xviii) the maximum allowable bit rate over the the transmission channel 26 is Rc 

In order to maintain equal video quality among all of the encoders 22, the 
target number of bits for each picture is prescribed to be directly proportional to the 
complexity of the picture, through a proportionality parameter C(l>. This constraint is 
30 defined by the following equations (1-a)- (1-c): 



(1-a) 
(1-b) 



T,i(t) = C(t) X X,i(t); 
Tpi(t) = C(t) X Xpi(t); and. 
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(1-c) TBi(t) = C(t) X XBi(t), 

where C(t) is independent of i. C(t) can vary with time and depends on the content of the 
input video signals Vi. At time t, an estimated encode rate ERi(t) for an encoder 22i can be 
5 calculated in accordance with the following equation (2): 

(2) EJt) = C(t)x[XH(t) + (Ni(t)/M;(t)-l)xXp,(t)4-(NKt)-Ni(t)/Mi(i))xXBi(t)]xFi(t)/N 

A counter (not shown) is initialized to a value of RcZ (where Z is a 
10 prescribed lime constant, e.g., 10 seconds) and incremented by 1 bit every 1/Rc second. 

Starting at a time t^ when any one of the L video encoders 22 begins to encode a picture, the 
counter is decremented by the target number of bits used to encode the picture. When the 
encoding of that picture is complete, the counter is adjusted for the discrepancy, if any, 
between the target number and actual number of bits used in encoding that picture. At time 
15 t, the value of the counter is denoted Count(t). A constraint defined by equation (3) below is 
imposed on ERi(t): 

L 

(3) E ERi(t)= Count(t)/Z 
/=1 



20 Equations (2) and (3) are used to compute a value for C(t) which is then used to compute 
T,i(t), Tp,(i), and T^SX) rising equation (1). ' 

The following equation (4) has be satisfied for all values of t and i, in order to prevent 
decoder buffer underflow and overflow: 

25 

(4) ERi(t)j< J Ri(u)du^Bui + Ei(t) 

t 

In the presently preferred embodiment of the present invention, the 
controller 30 controls the encoding parameters of each of the video encoders 22, in 
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accordance with the control algorithm, in such a manner as to prevent buffer underflow and 
overflow for each of the L video encoders 22. Notably, the L encoders 22 are not required 

to be synchronized. 

The controller 30, in accordance with the control algorithm, generates 
control signals CSi which are applied via respective control signal lines (e.g.. a bi-directional 
control signal bus 50) to the control inputs C of the respective video encoders 22. These 
control signals CSi carry the values of the encoding parameters which are dynamically 
controlled by the controller 30, including the target number of bits for encoding each picture 
encoded by the encoders 22, the minimum and maximum number of bits for encoding each 
picture encoded by the encoders 22, and the bit rate (channel output rate) R, for each of the 
encoders 22. 

In this connection, the target number of bits communicated by the 
controller 30 to the encoders 22 is a recommended (target) number, and not an absolute 
requirement. The actual number of bits used by an encoder 22i to encode a picmre may vary 
from the target number of bits specified by the controller 30 for that picture, within a range 
between the minimum and maximum number of bits also specified by the controller 30 for 
that picmre. If the number of bits acmally used to code the picture is within this specified 
range, then it is guaranteed that no underflow or overflow of the buffer 37 of the 
corresponding decoder 33i will occur. 

The buffer 27 of each encoder 22i is emptied at the bit rate Rj specified 
by the respective control signal CSi from the controller 30 until the bit rate R. is changed, at 
which time, it cominues to be emptied, but at the new bit rate R.. Preferably, the bit rates R^ 
of all L of the encoders 22 are changed by the controller 30 simultaneously, by way of the 

control signals CSi. 

With reference now to FIG. 3, the concept of coding events and rate 
events will now be discussed in order to provide the necessary background for further 
detailed description of the present invention. In general, a coding event occurs at a time t if 
any of the encoders 22i encodes a picture at that instant. A rate event occurs at time t if a 
change in the bit (output) rates Ri of the encoders 22i is scheduled for that time. 

These concepts are illustrated in FIG. 3. which shows coding events 
corresponding to the times t,(l,8), t„(l,9), t,(l,10), t,(l,ll), and t,(1.12). at which an 
encoder "1" encodes pictures 8, 9, 10, 11. and 12, respectively, and the times t,.(i.78), 
t,(i,79), te(i,80). te(i,81), and te(i,82) that an encoder "i" encodes pictures 78, 79, 80, 81, and 
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82, respectively, and the times t,(L,217). t,(L.218), t,(L,219). and t,(L,220) that an encoder 
"L" encodes pictures 217, 218, 219, and 220, respectively; and, shows rate events 
corresponding to the times t, and t2. As is true in the case of encoder "L", the encode times 
need not be periodic, e.g., the encode times are not periodic in the case of pictures that have 
5 undergone a reverse three-two pulldown procedure. Non-periodic encode times can also 
occur in the case of pictures having interspersed frame and/or field structures. 

It will be readily recognized by those skilled in the pertinent art that more 
than one coding event can occur at the same time, and further, that one or more coding 
events and a rate event can occur simultaneously. 

10 The discussion will now be directed to a further detailed description of the 

control algorithm employed by the controller 30 in controlling the encoding parameters of the 
multiple encoders 22. The first aspect of the control algorithm that will be described is the 
initialization procedure which is performed at the start-up time of the multiplexed system 20. 
The second aspect of the control algorithm that will be described is the processing which is 

15 performed in the case of a coding event. The third aspect of the control algorithm that will 
be described is the processing which is performed in the case of a rate event. 

At system start-up time, all L of the encoders 22 are given default values 
X,i(0), Xpi(O), and XBi(O) for the estimates of the complexity of the next I, P. and B pictures, 
respectively, to be encoded thereby. The initial bit rate Rj(0) for each of the encoders 22 is 

20 either given to the controller 30 and the encoders 22 by a supervisory controller (not shown) 
of the system 20, or are all sec to the same default value, (0) = Rc/L. Of course, the 
default values X,i(0), Xpi(O), and XgiCO), the channel rate Rc, and the number of encoders L 
must be provided to the controller 30, e.g., either input or preprogrammed thereinto. 

When the next event is a coding event, the control algorithm must 

25 determine the target number of bits to be used by the encoder 22 which is going to encode 
the next picture corresponding to the coding event. In the case that the next event constitutes 
more than one simultaneous coding events, whereby more than one (e.g., two) of the L 
encoders 22 are going to encode different respective pictures (e.g., such as times 1^(1.9) and 
t^(L,219) in FIG. 3), the control algorithm preferably first computes the target number of bits 

30 to be used by one of the encoders in encoding the next picmres contained in the video signal 
Vi which it is processing, and then computes the target number of bits to be used by the 
other one of the encoders in encoding the next pictures contained in the video signal Vi 
which it is processing. As will become more fully apparent hereinafter, if a rate event 
occurs simultaneously with a coding event, then the control algorithm preferably computes 
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the target number of bits values for all of the encoders prior to computing the new (updated) 
encoder bit rates. 

The following example is given to illustrate the above. Namely, assume 
that the next coding event is the encoding of the next picture by an encoder 22i at time t . 
5 Prior to the encoding of this next picture to be encoded, the target number of bits T,i(t), 
Tpi(t), or TeXt). as the case may be, for this next picture to be encoded, is communicated to 
the encoder 22i by the controller 30. via the control signals CSi. The controller 30 
calculates these values using equations (1). (2). and (3) and the current values for X,|(t). 
Xpj(t). and XBi(t), with the constraint that equation (4) is also satisified. If equation (4) is not 
10 satisfied, then the target number of bits is clipped (truncated) to the closest value that will 
meet this constraint. In the following discussion, the constraints that are necessary for 
satisfying equation (4) are fully developed. 

In the ensuing discussion, the encode time for the last picture encoded by 
an encoder 22i will be denoted t,(i.k). the encode time for the next picture to be encoded will 
15 be denoted t,(i, k + 1). and the encode time for the picture following that will be denoted 
te(i,k+2). Since the next picture is to be encoded by encoder 22i at time r. then 
T =t,(i,k+ 1). For purposes of the present discussion, it will be assumed that if encoder 22i 
encodes picture j at lime t.(i.j), then all of the data that comprises the coded picture j will be 
inserted into the encoder buffer at time t,(i,j). 
20 If the number of bits used to code the picture j is b(i.j). then the encoder 

buffer status is defined as E.(t.(i.j)) = E.(t%(i.j)) + b(i,j). During the time interval t (where 
.r=^i,k+l))<.t<t,(i,k+2)), the encoder buffer status is dependent upon the initial buffer 
status at time r, the number of bits used to code the picture that is coded at timer, and the 
rate at which data is removed from the encoder buffer over that interval. 
25 In the presently preferred embodiment of the present invention, the 

computations based on equations (5) and (6) presented below are performed at the earliest 
possible time so as to minimize the possibility of any events occuring between times t and t. 
For example, with reference to the example illustrated in FIG. 3, if r corresponds to 
t,(1.10). then the computations based on equations (5) and (6) are preferably made as soon as 
30 possible after the lime t,. which is the last rate change event prior to the time t 

The constraints which must be satisfied are given by equation (4). and are 
calculated using the assumption that Ri(t) is constant in the time interval under consideration. 
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and is thus equal to R,(t) . However, it should be clearly understood that it is not necessary 
that Ri(t)actually remain constant over this time interval, since the controller 30. whenever it 
changes the bit rate Ri(t) will, in accordance with the control algorithm, automatically verify 
that the new value chosen for the bit rate R,(t) will satisfy ail constraints imposed by the 
5 control algorithm. One of these constraints is that for the time period T=te(i,k+1) <_ t < 
tj(i,k+2), equation (4) requires that the following equation (5) be satisfied: 

(5) /?,(T) .r A, - B,. < E.iT) < /?, X A,. 
10 since, 

t 

(6) Ei(t)= Ei(t%(i,k)) + b(i,k+l)- f Ri(v)dv, 

where b(i,k+ 1) is the number, of bits used by encoder i in encoding the picture that is 
15 encoded at time t,(i,k+ 1). 
te(i.k) 

The constraints on b{i,k + I) are; 
20 (7) R,(t)x(A,+t,(i.k+2)-t,(i,k+l)+ f Ri(v)dv-Bai-E,(t%(i.k))_<b(i.k+1) 

and, 

(8) b(i,k+l)<. Ri(r)xA,+ [ Ri(v)dv-Ei(t%(i,k)), 

25 
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where EiCl^.Xit'^)) is the number of bits in the buffer of encoder i just after all of the data 
corresponding to the klh picture has been loaded thereinto. 

Satisfying the constraint given by equation (7) ensures that decoder buffer 
overflow does not occur, and satisfying the constraint given by equation (8) ensures thai 
decoder buffer underflow does not occur. 

The prevention of encoder buffer underflow and overflow is managed in 
the following manner. More particularly, if the kth picture was the last picture encoded by 
encoder i, the encoder buffer status for the time interval t =te(i,k+ 1)_< t < ie(i,k + 2) is given 
by the following equation (9): 

(9) E,(t) = E,(t%(i,k)) + b(i,k+l)- J R,(v)dv-R.(r)x(t-T). 

The above equation (9) assumes that the rate at which the encoder buffer 
is emptied is constant over the period under consideration. In order to avoid encoder buffer 
underflow, the constraint on b(i,k-M) is: 

T 

(10) b(i,k+l)>LR.(r)xt,(i.k-f2)-i,(i,k-M)4-29 f R,(v)dv-E,(t\.(i,k)). 
In order to avoid encoder buffer overflow, the constraint on b(i,k+l) is: 

T 

11) b(i.k+l)^ f Ri(v)dv4-B,,i-Ei(t^(i,k)). 

If the target number of bits T,i(t), Tpi(t), or T^M computed using 
equations (l)-(3) does not satisfy the constraints given by equations (7) and (8), then the 
value closest to the target value that does satisfy the constraints given by equations (7) and 
(8) is chosen. This value will be either the upper limit given by equation (8) or the lower 
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limit given by equation (7), The upper limit given by equation (8) constitutes the maximum 
value for T,i(t), Tpi(t), or Te^t), and the lower limit given by equation (7) constitutes the 
mimimum value for T,i(t), Tp,(t). or TgiCt), thereby establishing a range of actual number of 
bits used to encode a given picture. As previously indicated, the control signals CSi issued 

5 by the controller 30 carry the values of the encoding parameters which are dynamically 
controlled by the controller 30, including the minimum and maximum number of bits for 
encoding each picture encoded by the encoders 22, Thus, this information is communicated 
to the encoders 22 by the controller 30. 

Further, the constraints given by equations (10) and (11) must be satisified 

10 in order to ensure that no encoder buffer underflow or overflow occurs. These computations 
can advantageously be performed by the encoders 22 themselves, although this is not limiting 
to the present invention. Optionally, the encoder buffers can be allowed to underflow, since 
it is possible to dynamically reallocate the thusly freed-up bandwidth to one or more other 
encoders. However, this is possible only if the next scheduled bit rate change occurs prior 

15 to the point in time when underflow is likely. Thus, if the next scheduled bit rate change is 
at time/rand/r <t,(i,k-l-2), then equation (10) can be solved by substituting/Tforte(i,k-f 2). 

As previously discussed, starting at a time t^i when any one of the L video 
encoders 22 encode a picture, the counter is decremented by the target number of bits used 
to encode the picture. When the encoding of that picture is complete, the counter is adjusted 

20 for the discrepancy, if any, between the target number and acmal number of bits used in 
encoding that picture. More specifically, Count(i) is then adjusted for any discrepancy 
between the target number and actual number of bits used in encoding that picture. In this 
connection, each of the encoders 22 conmiunicace the acmal number of bits which they 
utilize in encoding each picture to the controller 30, e.g., via the bidirectional bus 50. 

25 Further, each of the encoders 22 also communicates to the controller 30 the estimate of the 
complexity X,i(t), Xpi(t), or XBi(t) of each picture and the picture type (I, P, or B) as soon as 
it is able to determine the same, e.g., via the bidirectional bus 50. This information is then 
used by the controller 30 in updating the solutions to the equations (1) and (2). In general, 
each of the encoders is programmed to communicate at least the following information to the 

30 controller 30, in accordance with the control algorithm used in the presently preferred 
embodiment of the present invention: 

a) the value of D,(t)\ 

b) the encode times x^ , for the next two pictures to be encoded; 
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c) the frame rate; 

d) the actual number of bits used to encode the last picture; 

e) the estimate of complexity X,i(t), Xpi(t), or X^,W for the last picture; 

f) the current value of Ni(t); and, 
5 g) the current value of Mi(t). 

The bit rate R^t) at which each encoder buffer is emptied is adjusted 
periodically, with all encoder bit rates Ri(t) being adjusted simultaneously. The bit rate 
adjustment period is preferably every 1/F seconds, where F is the maximum frame rate of 
the input video Vi to any one of the L encoders 22. Prior to making a bit rate adjustment, 
10 the below-described procedures and computations are made by the controller 30 in 

accordance with the control algorithm of the presently preferred embodiment of the present 
invention. 

More particularly, each one of the L encoders 22 communicates to the 
controller 30 its buffer status E/t) upon completing the encoding of each picture. The 

15 controller 30 stores this buffer status information for each ith one of the encoders 22 

("encoder i") for a period of A.seconds. FIG. 4 illustrates the change of the output or bit 
rate Ri(t) of the encoder i. The dotted vertical line denotes a time instance when the next 
event is a (bit change) rate event, and t is the time that the rate event is scheduled to occur. 
As shown in FIG. 4. the bit rate R^O of the encoder i is changed by the controller 30 to 

20 Ri(r*) at the rate change time t, FIG. 4 also depicts a number of prior bit rate changes for 
the encoder i. where the kth picmre was the last picmre encoded by the encoder i prior to the 
illustrated rate event, and the k-f 1st picture is the next picture to be encoded by the encoder 
i after the bit rate change is effected. 

In order to ensure that the lower inequality of equation (4) is satisfied, the 

25 constraint represented by the following equation (12) must be satisified: 

O^A.) 

(12) Ej(t)<. J Ri(v)dv vr where 7-A^<i<,T. 

t 

Let t be such. that t,(i.j)^t<te(ij + l). If t is within the range given in 
30 equation (12), then for the example depicted in FIG. 4, j would take values from (k-3) to k. 
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The encoder buffer status is given by the following equation (13); 



t 

(13) Ei(t)= Ei(t\(iJ))- [ Ri(v)dv. 

ij) 



ii(t\(ij))- I 



5 In order to ensure that decoder buffer underflow does not occur, a 

constraint is imposed that the new (updated) rate R,(r*) satisfy the following equation (14): 

T 

(14) Ri(T^)^Max[E,(t\.(ij))- f R,(v)dv/t,(i,j) + A.-r] 

10 where the maximization is done over all values of j where j is such that r-A. <tc(ij)j< t. In 
the example illustrated in FIG. 4, these values of j would be (k-2), (k-1), and k. Note that if 
the numerator of the expression of equation (14) is negative, the rest of the expression need 
not be computed. Also, if the numerator is negative for certain values of j, those values of j 
need not be considered when this expression is computed in the future for further rate 

15 changes. If the numerator is negative for a certain of t and j, it will be negative for that 

value of j and all greater values of r. The constraint given by equation (14) provides a lower 
bound on the new (adjusted) bit rate for encoder i. If the bit rate is any lower bound, there is 
a risk of decoder buffer underflow. 

20 In order to ensure that decoder buffer overflow does not occur, a 

constraint is imposed that the following equations (15) and (16) are satisfied: 



(15) j Ri(v)dv^Bd.i(t) + E,(t) vr where r-A.^tj< r 
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(16) Ri(T-)>.Min[EXt%(i.j))+B,, - j Ri(v)dv/t,('i.j + 1)+ V^J- 



where the minimizaiion function is applied over all values of j where j is such that both of 
the following conditions are satisfied: (1) t.(i.j)<.T.and(2)t,(i.j)> r-A, . In the example 
5 illustrated in FIG. 4, this would correspond to values of j that range from (k-3) to k. 

Thus, in determining the new (adjusted) bit rate Ri(T-) for encoder i. 
satisfying the constraints given by equations (14) and (16) will ensure that no decoder buffer 
underflow or overflow will occur. 

The encoder buffer state from time t (the time of the next scheduled bit 

10 rate change) to time/r is given by the following equation (17): 



(17) E(t)= E,(t\{i,})) + T [b(i.k+j)Xu(t-t,(i,k+j))]- f Ri(v)dv-Ri(r-)x(t-T)). 

y=l f,(«.*) 



where u(t) is the step function with value 1 when t >. 0 and value 0 when t < 0. 
,5 In order to ensure that encoder buffer overflow does not occur, a 

constraint is imposed that the following equation (18) is satisfied: 



(18) Ri(r')< Min[E,(t%(i.k)) + 52b(i,k-<-j)- f R,(v)dv/t,(i.k + n+ D-t], 

; = 1 tfi^k) 



20 where the minimization is done over all values of n where n ^ 0. and t,(i.k + n)<./T . If 
ai,k + n+l)>:/T. then t,(i.k + n+l) in equation (18) is replaced by/r. If equation (18) is 
satisfied, it is ensured that no encoder buffer inderflow will occur prior to time/r . Tlie 
possibility of encoder buffer underflow occuring after time/r can be handled at time/r by 
changing the bit rate in such a manner as to prevent encoder buffer underflow. 



wo 98/11687 



PCT/IB97/01085 



23 

Equation (17) can be used to derive the following equation (19) which is 
imposed as a constraint in order to avoid encoder buffer overflow: 



(19) 



R,{r)>_ Max fE,(t%(i.k))' 



b(i.k+j)- 



Ri(v)dv-B,A(i.k + n)-T], 



where the maximization is done over all values of n such that n is a positive integer and 
t,(i,k + n)<./r. If equation (19) is satisfied, it is ensured that encoder buffer overflow will not 
occur prior to time/r. The possibility of encoder buffer underflow occuring after lime/r can 
be handled at time/r by changing the bit rate in such a manner as to prevent encoder buffer 
10 overflow. 

The encoder bit (output) rates Ri(t) are computed by the controller 30 in 
accordance with the following algorithm. Namely, equation (14) is used to calculate the 
minimum bit rate R.„i„,j(i) required for each encoder 22 in order to prevent decoder buffer 
underflow. Equation (16) is used to calculate the maximum bit rate Rna,,d(i) for each encoder 
15 22 required in order to avoid decoder buffer overflow. Equation (18) is used to calculate the 
maximum bit rate R,™,..(i) for each encoder 22 required in order to prevent encoder buffer 
underflow. Equation (19) is used to calculate the minimum bit rate Rn,m.e(i) for each encoder 
22 required in order to prevent encoder buffer overflow. 

In accordance with the presently preferred embodiment of the present 
20 invention, the control algorithm employed by the controller 30 maintains all of the encoder 
buffers 22 as empty as possible in order to provide increased flexibility to the rate allocation 
procedure, and to optimize the same. 

The problem of maintaining the encoder buffers 22 as empty as possible 

can be stated in the following manner. 



25 



Namely, determine Ri(T*) for i = 1 to L, such that 



(20-a) Ri(7-) >. R^„.o(j). 

(20-b) Ri(T*) >. R™,..c(i), 

30 (20-c) Ri(T') <. R™,.d(i). 
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(20-d) R,(7-) <. Rm,;,,p(i). and 

(20-e) minimize Max[Ei(/T)]. 

The following algorithm provides the optimal solution to this problem. 

5 

Step 1: 

Define Roe,„(i) = R>,m.e(i) - ^Xt'). 

L 

10 Define R„„ = Rc - E_Ri(T*) 

i = 1 

Perform the following computations: 

Rmir(i) = Min[R„,„,e(i), Rmin.u(i)] 

15 R™.(i) = Min(R^.e(i), R™x.a(01 
L 

Rmax = E 



Initialize Rj( t' ) = R™,.e(i)-Rn>ox-Rc/L. 
Compute Rdeh.(i)- 

20 

Step 2: 

Perform each of the following operations for all values of i: 
IfR,(T*) < R„^„(i). 
25 then set R^r*) = R„rt„(i). 
IfRi(r) > R™,{i). 
then set Ri(T') = R„«,x(0- 
Compute Rjeiia(i)- 
Compute R,ei,. 

30 
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Step 3: 

If R,ert = 0. then optimality has been achieved. Terminate procedure. 
If R,^,i < 0. then, among those encoders 22 in which RjCr*) >R,„„(i), select the one(s) with 
5 the lowest RjeiJO. This set of encoders is denoted C, Next, decrease the R,(t*) and update 

and R,cu of all of the encoders in set C by the same amount until one of the following is true: 

(a) there are other encoders that have the same R^ciiaCO as the encoders in set C; or, 

(b) Rj(T*) = Rmm(i) for one of the encoders in C; or, 
10 (c) R,„, = 0. 

Repeat Step 3. 

If Riea > 0, then, among those encoders 22 in which R,{r) > ICixCO, select the one(s) with 
15 the lowest RddJi). This set of encoders is denoted C. Next, decrease the R,(t^) and update 

Rdclta(0 

and R,^rt of all of the encoders in set C by the same amount until one of the following is true: 

(a) there are other encoders that have the same Rdeita(i) as the encoders in set C; or, 

(b) Ri(r*) = Rrvn^CO for one of the encoders in C; or, 

20 (c) R,„-, - 0. 

Repeat Step 3. 

Although the present invention has been described in detail hereinabove, it 
25 should be clearly understood that many variations and/or modifications of the basic inventive 
concepts taught herein which may appear to those skilled in the pertinent art will still fall 
within the spirit and scope of the present invention, as defined in the appended claims. 
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Claims: 



1 A multiplexed data transmission system, including: 

a plurality of encoders each having a data input coupled to a respective one of a 
plurality of data signals, a control input, and an encoder output; 

a multiplexer having a plurality of multiplexer inputs coupled to respective ones 
5 of said encoder outputs, and a multiplexer output coupled to a transmitting end of a data 
transmission chatmel; and, 

a controller for producing control signals in accordance with a control 
algorithm, said control signals being coupled to said control input of respective ones of said 
encoders for dynamically controlling at least two encoding parameters of said encoders, a 
10 first one of said encoding parameters being an output bit rate of each of said encoders. 

2 The system as set forth in Claim 1. wherein said multiplexer multiplexes 
said encoder outputs and outputs a single, multiplexed encoded data stream over said data 
transmission channel at a transmission channel bit rate which is less than or equal to a 
prescribed maximum transmission channel bit rate. 

J 5 3 The system as set forth in Claim 2, wherein a second one of said 

encoding parameters comprises a target number of bits used by each of said encoders to 
encode each of a plurality of prescribed blocks of data contained in respective ones of said 
data signals. 

4 The system as set forth in Claim 3. wherein said transmission channel bit 
20 rate is equal to the sum of said output bit rates of all of said encoders. 

5 The system as set forth in Claim 4, wherein said control algorithm 
ensures that the quality of encoded data represented by all of said encoder outputs is 
substantially equalized without violating a data encoding and transmission protocol utilized by 
the system. 

25 6. The system as set forth in Claim 5, further comprising a plurality of 

encoder buffers associated with respective ones of said encoders, wherein said comrol 
algorithm imposes no constraints on the allowable size of said encoder buffers. 
7 The system as set forth in Claim 6. wherein said control algorithm 

imposes no constraints on the allowable size of decoder buffers used in a receiving station 
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coupled to a receiving end of said data transmission channel. 

8. The system as set forth in Claim 7, wherein said control algorithm 

imposes no constraints on the allowable relative sizes of said encoder buffers and said 
decoder buffers. 

5 9, The system as set forth in Claim 8. wherein said control algorithm 

minimizes the number of bits contained by each of said encoder buffers. 
10. The system as set forth in Claim 6, wherein said data signals comprise 

video signals, and said prescribed blocks of data comprise successive pictures contained in 
respective ones of said video signals, 
10 11, The system as set forth in Claim 10, wherein said data encoding and 

transmission protocol comprises an MPEG protocol. 

12. The system as set forth in Claim 7, wherein said control algorithm 
ensures that no underflow or overflow of either said encoder buffers or said decoder buffers 
occurs, even in the event that a failure mode occurs in which said controller is unable to 

15 dynamically control said output bit rate of each of said encoders. 

13. The system as set forth in Claim 6, wherein said control algorithm 
imposes a constraint that no underflow or overflow of said encoder buffers can occur. 

14. The system as set forth in Claim 13, wherein said control algorithm 
imposes a further constraint that no underflow or overflow of decoder buffers used in a 

20 receiving station coupled to a receiving end of said data transmission channel can occur. 

15. The system as set forth in Claim 14, wherein said data signals comprise 
video signals, and said prescribed blocks of data comprise successive pictures contained in 
respective ones of said video signals. 

16. The system as set forth in Claim 15, wherein said target number of bits 
25 are allocated by said controller in accordance with said control algorithm in such a manner 

that all of said encoders are allocated a target number of bits functionally related to a 
measure of complexity of a picture currently being encoded thereby. 

17. The system as set forth in Claim 15, wherein said control algorithm 
imposes an additional constraint that an actual number of bits used by each of said encoders 

30 in encoding each of said pictures not be greater than a dynamically allocated maximum 
number of bits and not be less than a dynamically allocated minimum number of bits. 

18. The system as set forth in Claim 17, wherein said controller dynamically 
varies said output bit rate of each of said encoders simultaneously at prescribed bit rate 
change intervals, in accordance with said control algorithm. 
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19^ The system as set forth in Claim 18, wherein said controller dynamically 

varies said target number of bits used by each of said encoders on a picture-by-picture basis, 
in accordance with said control algorithm. 

20. The system as set forth in Claim 19, wherein each of said encoders 

5 communicates to said controller the actual number of bits used thereby in encoding each 
picture encoded thereby. 

21. The system as set forth in Claim 20, wherein each of said encoders 
communicates to said controller a measure of complexity of each picture encoded thereby, 

22. The system as set forth in Claim 17, wherein said controller dynamically 
10 varies said output bit rate of each of said encoders simultaneously every 1/F seconds, in 

accordance with said control algorithm, where F is a maximum frame rate of any of said 
video signals. 

23. The system as set forth in Claim 17, wherein said control algorithm 
includes the following equations: 

15 T,(t) = C(t) X X.Xt); Tp,(t) - C(t) x Xp^W; and, = C(t) x XeXt). 

24. The system as set forth in Claim 23, wherein said control algorithm 
further includes the following equations: 

E,i(t) - C(i) X [X,(t) + (N,(t)/Mi(t) - 1) X Xp,(t) -h (NXt) - N-.(t)/M,(t)) x XBi(t)] x Fi(t)/H(t); 
and, 
L 

20 52 ERi(t)= Count(t)/Z. 
/ = l 

25. The system as set forth in Claim 24, wherein said control algorithm 
further includes the following equation: 

ERi(t) < J Ri(u)du <.B,i + Ei(t) . 
r 

26. The system as set forth in Claim 25, wherein said control algorithm 
25 further includes the following equations: 

Ri( T )x A' -B,i^Ei(t)^RX r )x A . ; 

t 

Ei(t)= E.(t%(i,k))-hb(i,k-M)- [ Ri(v)dv; 
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Ri(t)x(A,+te(i,k+2)-t,(i,k+l)+ f Ri(v)dv-Bd,-Ei(t*e(i.k))<.b(i,k+1); 
b(i,k+l)<.Ri(r)xA,+ f Ri(v)dv-Ei(t%(i,k)); 



Ej(t) = E.(t%{i.k))+b(i.k + l)- | Ri(v)dv-Ri(T)x(t-T); 

T 

b(i,k+l)>:Ri(T)xt,(i,k+2)-t,(i,k+l)+ ( Ri{v)dv-E,(t%(t,k)); 



T 

5 b(i,k+l)<. J Ri<v)dv + B,,-Ei(t%(i.k)); 



t+A) 

Ei(t)^ j Ri(v)dvv/ where T-A,.j<t^T;and, 



E,(t)= E,(t\.(i.j))- ( Ri(v)dv. 



(i.j))- J Ri( 



27. The system as set forth in Claim 26, wherein said control algorithm 

further includes the following equations: 



T 

10 Ri(T')^Max[Ei(t%(i.j))- ( RXv)dv/t,(i,j) + A.-t]; 



t*c(i.j))- j 



I Ri(v)dv^Bj.,(t) + Ej{t)vrwhereT-A, <.t<T; 
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T 

Ri(r)>.Min(E,(t%(i,j))+Bj,- ( Ri(v)dv/t,(i,j + l) + A,.-t']; 

ij) 



t 

t%(i,j))+Bj,- J 



Ei(t) = Ei(t\.(i.j)) + y: [b(i,k+j)Xu(t-te(i,k+j))]- Ri(v)dv-R,(T-)x(t-T)); 



R.(T-)<.Min[Ei(t%(i,k))+X b(i.k+j)- Ri(v)dv/ai.k + n+l)-T]; and. 

> = 1 tjCi,k) 



Ri(T')>.Max[E,(t%(i,k)) + X; b(i.k+j)- [ Ri(v)dv-Be.A(i,k+n)-T]. 

y=l tfi,k) 

5 28. A method for dynamically controlling a multiplexed data transmission 

system which includes a plurality of encoders each having a data input coupled to a 
respective one of a plurality of data signals and each producing a respective encoded data 
output, the method including the steps of. 

multiplexing the respective encoded data outputs from all of the encoders 
10 to produce a single, multiplexed encoded data stream; 

coupling said multiplexed encoded data stream to a transmitting end of a 

data transmission charmel; and, 

dynamically controlling at least two encoding parameters of the encoders, 
a first one of said encoding parameters being an output bit rate of each of the encoders. 
15 29. The method as set forth in Claim 28, further comprising the step of 

outputiing said multiplexed encoded data stream over said data transmission channel at a 
transmission channel bit rate which is less than or equal to a prescribed maximum 
transmission channel bit rate. 

30. The method as set fonh in Claim 29, wherein a second one of said 

20 encoding parameters comprises a target number of bits used by each of the encoders to 
encode each of a plurality of prescribed blocks of data contained in respective ones of the 
data signals. 

31 The method as set forth in Claim 30, wherein the dynamically controlling 
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step is carried out in accordance with a control algorithm which ensures that the quality of 
encoded data represented by all of the encoder outputs is substantially equalized without 
violating a data encoding and transmission protocol utilized by the system. 

32. The method as set forth in Claim 31, wherein: 

5 the system further includes a plurality of encoder buffers associated with 

respective ones of the encoders; and, 

said control algorithm imposes no constraints on the allowable size of said 

encoder buffers. 

33. The method as set forth in Claim 32, wherein said control algorithm 
10 imposes no constraints on the allowable size of decoder buffers used in a receiving station 

coupled to a receiving end of said data transmission channel. 

34. The method as set forth in Claim 33. wherein said data signals comprise 
video signals, and said prescribed blocks of data comprise successive pictures contained in 
respective ones of said video signals, 

15 35. The method as set forth in Claim 34, wherein said data encoding and 

transmission protocol comprises an MPEG protocol. 

36. The method as set forth in Claim 33, wherein said control algorithm 

imposes a constraint that no underflow or overflow of either said encoder or decoder buffers 
can occur. 

20 37. The method as set fonh in Claim 34, wherein said target number of bits 

are dynamically allocated in accordance with said control algorithm in such a manner that all 
of the encoders are allocated a target number of bits proportional to a measure of complexity 
of a picture currently being encoded thereby. 

38. The method as set forth in Claim 37, wherein said control algorithm 

25 imposes a constraint that an actual number of bits used by each of the encoders in encoding 
each of said pictures not be greater than a dynamically allocated maximum number of bits 
and not be less than a dynamically allocated minimum number of bits. 

39. The method as set forth in Claim 38, wherein the dynamically controlling 
step includes the further sub-step of dynamically varying said output bit rate of each of the 

30 encoders simultaneously at prescribed bit rate change intervals, in accordance with said 
control algorithm. 

40. The method as set forth in Claim 39, wherein wherein the dynamically 
controlling step includes the sub-step of dynamically varying said target number of bits used 
by each of the encoders on a picture-by-picture basis, in accordance with said control 
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algorithm. 

41 The method as set forth in Claim 33, wherein said control algorithm 

imposes a constraint that no underflow or overflow of either said encoder or decoder buffers 
can occur, even in the event that a failure mode occurs in which said output bit rate of each 
of said encoders can not be dynamically controlled. 
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(57) Abstract 

A method and apparatus for dy- 
namically allocating the available band- 
width of a common transmission channel 
of a multiplexed system among multiple 
encoders in such a manner as to maxi- 
mize and equalize the quality of the en- 
coded data output by all of the encoders, 
while also preventing underflow or over- 
flow of encoder or decoder buffers at 
each end of the common transmission 
channel, and moreover^ while also ensur- 
ing compliance with (i.e.» without violat- 
ing) the data encoding and transmission 
protocol utilized by the system. Further, 
the bandwidth of the common transmis- 
sion channel is preferably allocated us- 
ing an algorithm that does not impose 
any constraints on the size of the en- 
coder or decoder buffers, other than any 
constraints specified by the data trans- 
mission protocol employed in transmit- ^22 
ting the encoded data over the common 

transmission channel. In a presently pre- . . , . .k r u i-^- 

ferred embodiment in which video signals from multiple sources are encoded, both the output channel rate ("bit rate ) of each of the encoders 
and the target number of bits for each pichire that is encoded by each encoder are controlled by a controller in accordance with a control 
algorithm which ensures that the quality of the encoded pictures output by all of the encoders is equalized and maximized, that no underflow 
or overflow of the encoder or decoder buffers occurs, and that the data encoding and transmission protocol utilized by the system is not 
violated Further the control algorithm employed by the controller preferably docs not impose any constraints on the sizes or relative sizes 
of the encoder or decoder buffers. Moreover, the control algorithm employed by the controller preferably maintains all of the encoder 
buffers as empty as possible in order to provide increased flexibility to the rate allocation procedure. 
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